Business Research in Eight Business Disciplines

This study recorded nearly 1.5 million citations to measure research productivity of the 4,918 full time faculty members with doctoral degrees at 51 leading US business schools. These schools had been included at least once in the 25 most recent ranking lists produced by three major business publications. This research included lifetime citation counts for each faculty member, and resulted in 1,497,162 citations that were recorded between March and June 2003. The citation counts were cumulated by academic discipline. The disciplines for which rankings were made were accounting, economics, finance, information systems, marketing management science, organizational behavior, and strategy. Ranked lists of the top 25 schools in each disciple are included. The paper contains a review of the literature on citation analysis, and suggests how citation analysis might be used as an assessment tool by business school administrators, professors, students, and corporate managers. INTRODUCTION Measuring the quality of academic faculties is a task that is both easy and difficult. It can be extremely easy, because one has only to ask the apparent leaders in the academic discipline, record their assessments of the reputations of various faculties, and draw conclusions based on their opinions. This can be done in a single day. A more rigorous approach is to examine quantitative factors that might go into the recipe for reputation. Chief among them is the number of publications. The implicit assumption is that an author with a great number of publications has his or her work read very frequently by others in the field, and that the more often a n author is read, the more impact he or she has on the thinking of others. The difficulty with this line of reasoning is that it is based on quantity of production, because only the number of articles is tallied. There is no attempt to measure whether the articles have influence or, in fact, whether they are even read by anyone. It could be, for example, that an author might have published dozens of articles, but none of the articles were ever used by anyone else to shape their thinking. In that case, an author rated fairly high in production quantity would be rated very low on measures of impact on others. Impact measures would be very valuable, but they are elusive. It would not seem to be an easy task to determine the impact of the 4,918 full time faculty members with doctoral degrees at 51 leading US business schools. That task was accomplished using citation analysis, however, and the results are presented here as a series of rankings of the productivity of each school's business research in eight academic disciplines. BACKGROUND ON CITATION ANALYSIS Citation analysis is a process that measures the number of times a published article has been referenced in other articles worthy of publication. The overall concept is to regard an article as having impact if a subsequent author deems it important enough to merit citing it in her own published work. Authors, articles, and journals with the most citations can logically be considered to have had the most impact. The process can be used to measure that impact of individual articles, the impact of the authors of those articles, the impact of institutions, or the impact of groups of articles. In this particular case, it measures the impact of one school's faculty in a given discipline. Citation analysis became an accepted method for analysis of research quality in the natural sciences in the early 1960s following the introduction of the Science Citation Index (SCI). In 1969, the Social Science Citation Index (SSCI) became a new source of information about the citation practices of major journals in the social sciences. Since that time the number of social science journals included in SSCI has grown to about 1,700, with many of the journals including all citations dating back to 1975. A citation is recorded when an article in one of the 1,700 journals indexed by SSCI has referenced a previous work, and that previous work is credited with the citation. Citation analysis is an established procedure for the analysis of contributions to knowledge, dissemination of knowledge, and extent of knowledge exchange in a given field (Garfield 1979), and is a way of tracing the development of thought in a discipline. Citation analysis can be used for several purposes, including evaluation of scientists, publications and institutions, investigation of hypotheses concerning the history or sociology of science and technology; and in the study of information search and retrieval procedures. (Peritz 1992). Citation analysis has several advantages over other methods of evaluating research productivity in that it is objective, quantifiable, and a logical measure of quality. It is evidence that an article has not only been published, but has been read and referenced by someone else in an article good enough to be published in a journal included in the SSCI. The reference provides evidence of influence on the author who referenced it. As a quantifiable indication of influence on the work of others, it is an objective measure of research quality. Citation analysis has also been correlated with scientific productivity and peer judgments of performance (Bayer and Folger 1966). Citation analysis is not without issues, and citation practices vary greatly by author. A critical review of citations analysis by MacRoberts and MacRoberts (1989) examined the issues in use of citation counts, including biased citing, self-citing, and difficulties in treating citations of multiple authors for an article. The article also noted variations in citation rate related to type of publication, nationality, time period, and size of specialty area. Low citation rates for many journal articles suggest much of published work in social science journals does not get read (Mahoney 1987, Hamilton 1991). The first known use of citations in the marketing discipline was the measurement of the impact of marketing scholars and institutions over the four-year period 1972-1975 (Robinson and Adler 1981). The accounting discipline began to use citation analysis to assess the impact of journals and articles in accounting research (Brown and Gardner 1985). In 1990, the Journal of Financial Economics provided citation index rankings, and Alexander and Mabry (1994) ranked journals based on the number of citations, and identified the 50 most cited journals and the 50 most-cited authors in finance for the time period 1987-1991. Borokhovich, Bricker, and Simkins (1994) studied 15,110 footnotes appearing in 685 articles in eight major finance journals for 1990-1991, and concluded that two journals provided the research core of finance research, most journals publish in a greater variety of areas than they have influence, and there was a low level of borrowing across disciplines. The applied operations research journal Interfaces used citations analysis to examine 1,294 articles, 2,194 authors, and over 2,500 citations over a 23 year period (Gupta 1997). Gupta found that about 48 percent of articles were never cited, and the average number of citations for those with at least one citation was 3.63. He also found that most citations occurred in the first three years after publication. Vincent and Ross (2000) provide perhaps the strongest recent review of the literature on citations analysis, and its uses and potential uses in business research. Both advantages and pitfalls are highlighted in their work. METHOD This study was designed to extend previous work done by authors who identified and ranked business schools based on measures of research productivity. Earlier work in this area was limited in scope, due primarily to a limited ability to access relevant data. For instance, most studies of business school research productivity focused on a specific discipline, or on a specific journal for a limited time period. The design of the present study benefits from recent advances in information technology. The researchers were able to gather all data needed for the study by using unobtrusive means. The study was divided into three phases. The first phase began by identifying the set of US business schools ranked in the top 30 at least one time by wellknown business school ranking systems (Business Week, US News and World Report, and Financial Times) over the period 1988 to 2003. The next step was to identify the faculty members to be included in the study, and to record the following data for each. Last Name, First Name, Middle Initial(s) Current School Current academic rank (Assistant, Associate, Full, Endowed Chair, Other) Current academic title (full title as listed on the school’s website) Ph.D. School, year, and discipline(s) Primary teaching/research discipline Secondary teaching/research discipline; School websites varied greatly in the amount of information available. For those instances when supplemental information was needed, the researchers used Digital Dissertations, internet search engines, and directories for faculty (Hasselback, 2002). The second phase of the study gathered citations data for each of the 4.918 faculty members identified in the first phase. The researchers used the Web of Knowledge to gather lifetime citations data from online data compiled and updated weekly by the Institute for Scientific Knowledge (ISI) from the SSCI. The endnote provides details about the process used to develop lifetime citation counts. All citation counts were captured between March 1 and June 30, 2003. The third phase ranked the 51 business schools in various ways using the citation count data. Total and average citation counts were developed for the faculty members of the 51 schools, so rankings could be generated for the 51 schools and for eight academic disciplines within the 51 schools. The resulting rankings based on citation counts were compared with rankings for the disciplines developed US News and World Report for perceived research quality. RESULTS The results were based on 1,497,162 citations, which was a snapshot of the lifetime citation counts for the 4,918 full time faculty members at the 51 business schools included in the study. The faculty members at these schools were then sorted into eight academic disciplines of interest, which were: accounting, economics, finance, information systems, marketing, management science, organizational behavior, and strategy. Ranked lists of the top 25 schools in each disciple were then developed. Table 1 is an alphabetical list of the 51 business schools included in the study. The 51 schools are all the schools that have been listed as being in the top 30 at least once by Business Week, US News and World Report, and Financial Times. There were 25 rankings lists generated by these publications between 1988 and 2003. The eight columns show the rank order for each university in a given discipline. The order is based on the mean number of citations for the business faculty within the discipline under examination. The mean number was chosen as the primary indicator, in order to more accurately reflect the research productivity of each faculty member. The table shows that a surprisingly large number of schools dominate the lists. Four schools were ranked within the top 30 for all eight disciplines (i.e. Chicago, Harvard, MIT, and Stanford) but nine schools were within the top 30 seven times, and nine additional schools were ranked six times. Tables 2 through 9 show rankings for faculties based on the mean number of citations per faculty member. The total number of citations is also shown, along with rankings on that measure, although ranking by total citations obviously favors schools with large faculties. Faculty size is given. There is a comparison with the ranking generated by US News and World Report in their 2003 update survey of the top schools in specialty areas of business for the six specialty areas that coincide with the eight under study. USNWR had surveyed deans and MBA directors at business schools to get “top 10” rankings for programs in those specialty areas . The specialty programs keyed to the table number are Accounting (2), Economics (3), Finance (4), Information Systems (5), Marketing (6), Organizational Behavior (7), and Strategy (8). In each case, the top 25 schools in terms of mean citations are shown.